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ABSTRACT 

Teaching with Technology (TWT) is a multi-year development 
program for Idaho teachers, funded and developed by the J.A. and Kathryn 
Alberston Foundation. TWT is a complement to the Opportunity 1 initiative 
that made educational technology available to Idaho schools. TWT provides 
intensive summer training workshops and offers support to teachers during the 
school year. TWT seeks to: increase teachers' knowledge and skills for 
working with new technology; and increase teachers' constructivist beliefs 
about teaching and learning. This report describes TWT ' s impact on teachers 
over the course of one academic year. Teachers' responses to a survey 
administered at three different times (i.e., prior to TWT summer workshops, 
immediately after the workshops, and ten months after the workshops) were 
examined. It was concluded that TWT had an impact on a large proportion of 
participating teachers — teachers reported substantial changes in their 
technology skills and in their beliefs about good teaching. It was also found 
that teachers participating in TWT very substantially increased their ability 
to use computer technology and somewhat shifted their conceptions of good 
teaching toward constructivism. Appended are: Teaching with Technology 
Survey, Wave 3; Who Participated in TWT and the Study?; Helpfulness of TWT 
Technology Fellows and Attitude Changes, by Wave 1 Pedagogical Beliefs and 
Technology Skills; Training Requests by Grade Level Taught, Changes from Wave 
1 to Wave 3; Overall Changes in Constructivist Beliefs, Between Waves, Paired 
T-test Results; and Overall Changes in Technology Skills, Between Waves, 
Paired T-test Results. (Contains 10 endnotes, 6 references, and 6 tables.) 
(MES) 
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Executive Summary 

Teaching with Technology (TWT) is a multi-year development program for Idaho 
teachers funded and developed by the J.A. and Kathryn Albertson Foundation. 

Teaching with Technology is a natural complement to the Opportunity 1 Initiative that 
made educational technology available to Idaho schools. 1 TWT provides intensive 
summer training workshops and offers support to teachers during the school year. TWT 
seeks to: 

• increase teachers’ knowledge and skills for working with new technologies, and 

• increase teachers' constructivist beliefs about teaching and learning 

This report describes TWT's impact on teachers' over the course of one academic year. 
We examined teachers' responses to a survey administered at three different times: 

Wave Administered ... 

1 Prior to TWT summer workshops 

2 Immediately after TWT summer workshops 

3 End of school year, 10 months after TWT summer workshops 

Our conclusions are based on a comparison of how teachers responded initially to the 
workshops, and what they indicated 10 months later, i.e., after teaching students in a 
real classroom for one school year. Survey items about teaching beliefs and technology 
skills are included in Appendix A. 2 

TWT had an impact on a large proportion of participating teachers. Teachers 
reported substantial changes in their technology skills and in their beliefs about 
good teaching. Immediately after the workshop there was a great deal of enthusiasm; 
many more teachers subscribed to constructivist beliefs than did before participating in 
the training. After 10 months of day-to-day classroom experience there was slightly less 
enthusiasm for constructivist pedagogy than there was immediately following the TWT 
workshop, but a greater proportion of teachers still endorsed constructivist than had 
before TWT participation. 

Teachers participating in TWT very substantially increased their ability to use 
computer technology and somewhat shifted their conceptions of good teaching 
toward constructivism. More importantly, in many cases TWT’s impact persisted after 
10 months of teaching. Indicators of program impact include: 

• Reports of the Helpfulness of TWT Technology Fellows and Attitude Changes 

• Changes in the Nature of Technology Training Requests Made by Teachers 

• Changes in Teachers’ Objectives for Computer Use with Students 

• Changes in Teachers’ Beliefs about Teaching and Learning 

• Increases in Teachers’ Technology Skills 
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Each of these success indicators is discussed in the following sections. We also 
examine which TWT participants reported more positive experiences, and which groups 
of teachers changed more than other groups. 

Reports of the Helpfulness of TWT 
Technology Fellows and Attitude Changes 

Overall response to TWT was very positive among participants in the study. 3 During 
Wave 3, we asked specifically about the TWT Fellows who had led the workshops and 
supported teachers during the school year. After 10 months in the classroom 71% of 
the teachers responding said the TWT Fellows were “extremely helpful” (Table 1); only 
5% said the Fellows were “not helpful.” 

Looking by grade level, Table 1 shows that elementary school teachers reported the 
most positive experiences, with 76% giving the most favorable responses to questions 
about the helpfulness of the Technology Fellows. Elementary school teachers also 
indicated that they had more positive attitudes about technology as a result of TWT. 
They were followed closely by middle school teachers, and a distant third was high 
school teachers, where only half responded that the Technology Fellows were 
extremely helpful (53%) and that they had changed their attitudes (56%) toward 
technology. 



Table 1. Percent of Teachers Reporting Positive Attitude Changes and 
“Extremely Helpful” Technology Fellows, byWave 1 Pedagogical Beliefs and Technology Skills 









Type of beliefs 


Type of skills 








Percent of 


Percent of 


Percent of 


Percent of 








teachers 


teachers 


teachers with 


teachers with 






Percent 


with less 


with more 


less advanced 


advanced 


Grade 




of 


constructivist 


constructivist 


technology 


technology 


Level 


Response 


teachers 


beliefs 


beliefs 


skills 


skills 




Attitude changed? 


71% 


72% 


69% 


83% 


60% 


All 


Extremely helpful? 


71 


73 


70 


71 


72 




N 


110 


49 


63 


52 


57 




Attitude changed? 


76 


72 


79 


89 


66 


Elem 


Extremely helpful? 


76 


69 


82 


75 


77 




N 


63 


29 


34 


28 


35 


Middle 

School 


Attitude changed? 


68 


67 


69 


69 


69 


Extremely helpful? 


72 


87 


59 


63 


86 


N 


29 


15 


16 


16 


13/14 


High 

School 


Attitude changed 


56 


83 


42 


89 


22 


Extremely helpful 


53 


60 


50 


75 


33 


N 


17 


6/5* 


12 


9/8* 


9 



Note: Wave 3 responses only. When there are two Ns the first indicates the number of responses to the “attitude 
changed” item and the second indicates responses to the “helpfulness” item. Otherwise, the lowest N is shown for 
each pair of items. 
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Teachers with less advanced technology skills said their attitudes toward technology 
changed more often than teachers with advanced technology skills (83% vs. 60%). In 
high schools, teachers in the low belief and low technology categories most frequently 
reported a positive attitude change and that the Technology Fellows were extremely 
helpful. Among middle school teachers, it was the teachers with higher technology 
skills (but still those with less constructivist beliefs) who said the Technology Fellows 
were most helpful. Among elementary school teachers, it was teachers with more 
constructivist beliefs who said this (82% compared to 69% of the teachers with less 
constructivist beliefs). 

When we combine the technology skills and belief categories into 4 categories 
(Appendix C) we see that the least positive responses are from those who started out 
with higher-than-average scores on one measure and lower-than-average scores on the 
other. Teachers who started with less constructivist beliefs and more technology skills 
least often changed their attitude toward technology. Of teachers who entered TWT 
with stronger constructivist beliefs and fewer technology skills, only 58% said the 
Technology Fellows were extremely helpful. 4 

Table 2 shows that teachers who reported the Technology Fellows were extremely 
helpful gained more technology skills than teachers who said the Fellows were only 
somewhat helpful. Those teachers who reported the TWT Fellows were not helpful 
made considerably less movement toward constructivist pedagogy and technology 
skills. 



Table 2. Standardized Residual Gain Scores for Technology Skills 
and Constructivist Beliefs 5 , by Helpfulness of Technology Fellows 



Helpfulness of 
Technology Fellows 


N 


Gain in 

Constructivist 

Beliefs 


S.d. 


Gain in 
Technology 
Skills 


S.d. 


Extremely helpful 


82 


.07 


.99 


.12* 


.93 


Somewhat helpful 


28 


-.01 


.79 


-.23 


1.08 


Not helpful 


5 


-.73 


1.66 


-.50 


1.35 


Total 


115 


.00 


1.00 


.00 


1.00 



* p < .10, Effect size = .35. 

Note. Standardized residual gain scores indicate how much scores were above or below what 
would have been predicted based on initial Wave 1 responses. A negative gain does not literally 
mean a loss of skills, but how much less of a gain than the average teacher. 



Changes in the Nature of Technology 
Training Requests Made by Teachers 

To see if TWT met teachers’ demands for training, we asked at each wave whether 
teachers wanted additional training on various computer topics. Choices for additional 
training were: “none”; “start from scratch”; “just a refresher”; and “advanced course.” 

The percent of teachers requesting training that would “start from scratch” decreased 
considerably over all three waves, indicating that the desire for initial training in each 
area was being met. At the same time, requests for “refresher only” and “advanced” 
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courses went up in several cases. This suggests that after increasing their basic 
technology skills, teachers still were interested in further training. 

Table 3 shows requests for further training concerning “managing students when 
integrating technology.” For this item, the percent of TWT teachers wanting to start 
from scratch dropped overall and the percent requesting a refresher increased. 
Appendix D shows how these responses differed by grade level taught. 



Table 3. Examples of Additional Training Requests, All Teachers 



Additional Training for 
Managing Students 
When Integrating 
Technology 


Wave 1 
teachers who 
did NOT answer 
Wave 3 
(N=230) 


Wave 1 
teachers who 
DID answer 
Wave 3 
(N=1 10) 


Wave 3 
teachers 
(N=1 10) 


Difference, 
for those who 
answered 
both waves 
(N=1 10) 


None 


5% 


4% 


13% 


9% 


Start from scratch 


43 


39 


13 


-26 


Just a refresher 


20 


20 


40 


20 


Advanced course 


32 


38 


35 


-3 



On most types of training requests, there was a substantial drop in the proportion of 
teachers requesting “refresher” or “start from scratch” courses. In some cases, there 
was also an increase in the proportion of teachers saying they desired no further 
training (“none”). In Wave 3, over half of the teachers for each grade level answered 
“none” for additional training in word processing, up more than 20% from Wave 1 
(Appendix D). 



For other types of training, there was a substantial drop in “start from scratch” requests, 
but an increase in the proportion of teachers wanting “advanced courses.” This is seen 
for teachers in secondary schools, and for these applications. 

• Digital imaging 

• Integrating technology in daily teaching 

• WWW as instructional resource 

The most demand for “starting from scratch” during Wave 3 concerned using databases 
(more than 20% for all grade levels), spreadsheets (24% of high school teachers) and 
creating multimedia (22% of high school teachers). 

From Wave 1 to Wave 3, elementary school teachers generally moved to requesting no 
additional training while secondary teachers generally moved to wanting advanced 
courses. For example, among teachers in high schools, there was a gain of 13% 
requesting advanced courses in spreadsheets, while for elementary school teachers the 
change was towards wanting no additional training. 

Changes in Teachers’ Objectives for Computer Use with Students 

In each Wave of the study, teachers indicated their top 3 objectives for using computers 
with students (Table 4). There was a substantial decrease in the proportion of teachers 
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choosing the more “traditional” objectives for student computer use including mastering 
or remediating skills, becoming better writers, and learning word processing. There was 
an increase in the percent choosing more constructivist-compatible objectives, 
particularly presenting information to an audience and working collaboratively. 

Table 4. Percent of Teachers Including Each Objective, in Their Top 3 



Type of Change 


Wavel 

% 


Wave2 

% 


Wave3 

% 




Teachers who 








did NOT 


Teachers who 








answer 


DID answer 








Wave 3 


Wave 3 






Increase 


(N=230) 


(N=1 10) 


(N=254) 


(N=1 10) 


Presenting Info to Audience 


28% 


39% 


48% 


48% 


Work Collaboratively 


21 


23 


47 


39 


Finding Ideas and Info 


68 


70 


77 


75 


Analyzing Info 


24 


15 


24 


27 


Communicating electronically with others 


5 


5 


8 


12 


Decrease 










Becoming better writers 


31 


34 


22 


25 


Mastering or remediating skills 


42 


29 


16 


22 


Word processing skills 


30 


34 


12 


17 


Little or no change 










Computer skills, other than word 










processing 


47 


47 


49 


44 



Interestingly, gains in some of these objectives continued from Wave 2 to 3 (unlike 
changes in teachers’ beliefs about teaching). Analyzing information and communicating 
electronically were placed among their top three objectives by greater proportions of 
teachers in each successive wave. 6 For example, 12% more teachers who answered 
both waves said “analyzing information” was a top objective in Wave 3 than in Wave 1 
and 16% more said this about working collaboratively. Presenting information and 
finding information had almost no decline in importance from Wave 2 to Wave 3. Other 
objectives reflecting constructivist pedagogy declined slightly from Wave 2 to Wave 3, 
but still showed an overall increase in importance from Wave 1 to Wave 3 (Table 4). 

Changes in Teachers’ Beliefs about Teaching and Learning 

For teachers participating in TWT, beliefs about teaching and learning changed in a 
constructivist direction, with the largest jump occurring immediately after the summer 
workshops. The change in responses on 8 separate belief questions was statistically 
significant and in a constructivist direction between Wave 1 and Wave 2 (Appendix E). 
Between Wave 2 and 3 there was considerable movement back in a traditional 
direction. Participants reported substantially less constructivist beliefs after 10 months 
of teaching than they did at the end of the summer. In particular, by Wave 3 more 
teachers were indicating the importance a “quiet classroom” than in Wave 2 (ES=.30, p 
< .01 ). Still, compared to Wave 1 , there were significant changes in a constructivist- 
compatible direction shown at the end of the year. There were also some constructivist 
beliefs that were endorsed more frequently in Wave 3 than in Wave 2, suggesting their 
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popularity did not drop but continued to rise. One of these beliefs included the 
importance of students establishing their own criteria for assessments. 

Among teachers in elementary and middle schools, a greater proportion of teachers in 
Wave 3 endorsed a constructivist approach as opposed to a structured instruction 
approach. Teachers in elementary schools also more frequently endorsed multiple 
simultaneous activities in the classroom in Wave 3 than in Wave 1 . In general, teachers 
in elementary schools changed their beliefs more. On the other hand, it was teachers in 
high schools who changed the most toward believing students should help set the 
criteria for assessing their work. 

Table 5 shows examples of beliefs where there was significant change between Wavel 
and Wave 3, by grade level taught. Regardless of grade level taught, teachers less 
often endorsed instruction based on giving students many easy problems to solve. 

Teachers who sustained their initial Wave 2 changes in a constructivist direction were 
elementary teachers, and teachers of smaller middle schools. Table 6 shows the 
change in scores between Wave 1 and Wave 3 for different subsets of teachers. 
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Table 5. Percent of Teacher Favoring Different Pedagogical Beliefs in Wave 1 and Wave 3, by Grade Level 



o 

ERLC 



All Grade Levels 


Wave 3 
teachers 


110 


55 

CM 

CO 


69 


69 


50 


▼— 


Wave 1 
teachers 
who DID 
answer 
Wave 3 


110 


48% 


to 


43 


36 


85 


Wave 1 
teachers 
who did 
NOT 
answer 
Wave 3 


230 


37% 


47 


44 


CO 


80 


High School 


Wave 3 
teachers 


00 


55 

CM 

CM 


72 


56 


39 


94 


Wave 1 
teachers 
who DID 
answer 
Wave 3 


00 


33% 


67 


CD 


50 


78 


Wave 1 
teachers 
who did 
NOT 
answer 
Wave 3 


42 


36% 


45 


09 


29 


74 


Middle School 


Wave 3 
teachers 


CO 


44% 


58 


75 


50 


oo 

oo 


Wave 1 
teachers 
who DID 
answer 
Wave 3 


CO 


53% 


50 


41 


32 


3 


Wave 1 
teachers 
who did 
NOT 
answer 
Wave 3 


00 

T?- 


39% 


tj- 


5 


27 


oo 


Elementary School 


Wave 3 
teachers 


5 


29% 


73 


70 


52 


92 


Wave 1 
teachers 
who DID 
answer 
Wave 3 


5 


49% 


48 


39 


33 


oo 


Wave 1 
teachers 
who did 
NOT 
answer 
Wave 3 


140 


CD 

CO 


50 




33 


co 




z 


Students 
should be 
given many 
easy problems 
to solve 


Multiple 

simultaneous 

learning 

activities 


Constructivist 

approach 


Comfort with 
constructivist 
instruction 


Students 
should set 
criteria for 
assessment 



IT 



oo 
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Changes in Teachers’ Technology Skills 



The chang/es in technology skills reported by teachers were even more substantial than 
the changes in teaching philosophy. Gains in technology skills were sustained between 
Wave 2 and Wave 3 better than changes in philosophy (Table 6). From Wave 1 
(beginning of year-long TWT program) to Wave 2 (end of one-week summer TWT 
training) changes in technology skills as indicated on 1 1 separate questions were 
statistically significant. 

The skills that increased the most during TWT were developing multimedia and creating 
web pages. There was some decline in these skills between the end of summer TWT 
training and the end of the school year. Appendix F highlights that while this did occur, 
all the evidence shows that teachers made substantial gains in skills that persisted after 
10 months. 

Which TWT Participants Changed the Most? 

Table 6 shows the original Wave 1 scores and how responses changed in Wave 3 for 
different subsets of teachers. It shows a small but statistically significant difference in 
beliefs (ES =.17, p < .001) and a large difference in skills (ES = 1.51, p < .001). 

Comparing the different grade levels, elementary school teachers started out with 
relatively low constructivist beliefs but by Wave 3 they had changed more than teachers 
in secondary schools and responded to items in a more constructivist fashion than 
teachers in other grade levels. Teachers in smaller middle schools started out with 
beliefs similar to teachers in larger middle schools, but the teachers in smaller middle 
schools reported more constructivist beliefs in Wave 3 (their scores averaged 3.67 
compared to 3.44 for teachers in larger middle schools). The increase in the 
constructivist beliefs of teachers in smaller middle schools is shown as an effect size 
gain of .40, p < .10. 

Across the different grade levels, teachers who started with less advanced technology 
skills ended with less constructivist beliefs than those teachers who began with more 
advanced technology skills, but they changed their beliefs more than those who started 
with advanced technology skills. The mean score of less technologically skilled 
teachers on the constructivist belief index in Wave 1 was 3.47 (compared to 3.55 for 
teachers with more technology skills) but they had an effect size gain of .24, p < .05. 

Every group of teachers showed major increases in technology skills, even after 10 
months of teaching. Teachers in high schools started with more technology skills. 

While they reported gains in technology skills, these gains were less than those 
reported by elementary and middle school teachers. By the end of Wave 3, elementary 
school teachers reported substantially more skills than high school teachers did in Wave 
1 (4.09 vs. 3.28), but still slightly less than high school teachers (4.25). Looking by 
grade level and size, we see teachers in larger schools started with more advanced 
technology skills, and gained more than teachers in smaller schools. 7 
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Table 6. Teachers Who Moved Toward Constructivist Beliefs and Gained 
Technology Skills, Paired-Sample Statistics 

Technology Skills 



Belief Index Index (higher 







(higher score = more 
constructivist beliefs) 


Effect 

Size 


score = greater 
technology skills) 


Effect 

Size 




N 


Wave 1 Wave 3 




Wave 1 


Wave 3 




ALL Teachers 


119 


3.45 


3.55 


.17*** 


3.16 


4.19 


1.51*** 






.58 


.60 




.68 


.72 




By Grade Level 
















Elementary 


63 


3.42 


3.60 


0.36*** 


3.05 


4.09 


1.66*** 


S.d. 




0.51 


0.65 




0.63 


0.73 




Middle school 


30 


3.33 


3.44 


0.17 


3.22 


4.27 


1.49*** 


S.d. 




0.66 


0.62 




0.70 


0.66 




High school 


18 


3.56 


3.51 


-0.07 


3.28 


4.26 


1.18*** 


S.d. 

By Grade Level and Size 




0.62 


0.52 




0.83 


0.74 




Middle school, smaller 


9 


3.41 


3.67 


0.40* 


2.89 


4.00 


1.42*** 


S.d. 




0.64 


0.52 




0.78 


0.79 




Middle school, larger 


19 


3.44 


3.44 


0.00 


3.41 


4.42 


1.60*** 


S.d. 




0.60 


0.60 




0.64 


0.56 




High school, smaller 


7 


3.56 


3.43 


-0.17 


2.96 


3.83 


0.93*** 


S.d. 




0.73 


0.60 




0.93 


0. 73 




High school, larger 


7 


3.75 


3.72 


-0.12 


3.66 


4.75 


2.21*** 


S.d. 




0.29 


0.35 




0.49 


0.18 




Within Grade Level Technology Skills Ntiles 
(Wave 1) 














Lower tech skills 


56 


3.31 


3.47 


0.24** 


2.60 


3.74 


2.84*** 


S.d. 




0.66 


0.66 




0.40 


0.65 




Higher tech skills 


61 


3.58 


3.65 


0.14 


3.67 


4.59 


2.09*** 


S.d. 




0.48 


0.54 




0.44 


0.50 




With Grade Level Constructivist Beliefs Ntiles 














(Wave 1) 
















Less constructivist beliefs 


49 


2.91 


3.12 


0.58*** 


2.98 


4.03 


1.52*** 


S.d. 




0.38 


0.49 




0.69 


0.81 




More constructivist beliefs 


67 


3.84 


3.86 


0.06 


3.29 


4.30 


1.55*** 


S.d. 

Starting Out (within grade level, Wave 1) 
with... 




0.34 


0.48 




0.65 


0.62 




Less technology skills, 
Less constructivist beliefs 


29 


2.80 


3.07 


0.66*** 


2.52 


3.57 


2.89*** 


S.d. 




0.41 


0.50 




0.36 


0.68 




Greater technology skills, 
Less constructivist beliefs 


20 


3.05 


3.24 


0.68** 


3.65 


4.69 


2.26*** 


S.d. 




0.27 


0.46 




0.46 


0.44 




Less technology skills. 
More constructivist beliefs 


26 


3.86 


3.90 


0.11 


2.69 


3.92 


2.87*** 


S.d. 




0.37 


0.53 




0.43 


0.58 




Greater technology skills. 
More constructivist beliefs 


41 


3.84 


3.85 


0.03 


3.68 


4.54 


1 .98*** 


S.d. 




0.31 


0.46 




0.44 


0.53 





*** p < .0001, ** p < .05, * p < .10 

Note . Effect size is based on Wave 1 standard deviations. The maximum score was 5.0 on both indices. The extent of the change 
is indicated by the effect size, and the statistical significance provides confidence that the result was not caused by chance alone. 
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Discussion 



In our analyses of TWT impact, we have focused on group averages and compared 
changes that occurred across different types of teachers. This is done to portray what 
happened to different groups of “average TWT participants.” In reality, however, people 
rarely embody the all the qualities attributed to the average person, and the reader is 
reminded that it is important not to assume that group averages always describe the 
pedagogy and technology skills or changes of individual teachers. Undoubtedly, there 
are some teachers within larger and smaller schools, regardless of grade level, who can 
provide examples of different types of pedagogical beliefs, technology skills, and 
changes. 

TWT participants who responded to the surveys were enthusiastic about their 
experiences. End-of-year responses about the helpfulness of the Technology Fellows 
could not have been much more favorable, and 95% of teachers said their attitude 
toward technology changed as a result of the TWT program. This proportion is notable 
given that such a large proportion of teachers already had considerable technology 
experience (Appendix A). 

More important than their initial enthusiasm is the ability of the TWT teachers to sustain 
that enthusiasm after 10 months in the classroom. The comparison between Wave 1 
and Wave 3 responses represent the best indication of the change that occurred 
overall. These analyses provide a very consistent indication that the training 
program has had a lasting and positive effect 

Findings comparing Wave 1 and 2 reflect an “enthusiasm effect" so that the changes in 
belief and skills that were reported immediately after the training were artificially high. 
Findings comparing Wave 2 and Wave 3, on the other hand, reflect a “reality effect” so 
that in many cases, not all, the initial changes could not be sustained in the face of the 
demands presented during the school year. Most notably, teachers by Wave 3 wanted 
quieter classrooms than they did at the end of the summer, and they were less 
confident in their skills to create a web page. 

Although the goals of TWT were not simply to increase teachers’ technology skills, it 
was easier to develop teachers’ technology skills than to change their beliefs 
about teaching and learning, and move them toward a more constructivist 
pedagogy. Teachers in elementary, middle, and high schools all reported gains in 
technology skills that persisted through the end of the year. In contrast, only teachers in 
elementary schools and in smaller middle schools reported sustained changes in 
pedagogical beliefs through Wave 3. 8 

The least change observed - and consequently, the biggest challenge - was in 
changing the beliefs of teachers in the larger secondary schools. However, there is 
some evidence of success for TWT even among teachers in these schools. These 
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different outcomes may be less an issue of program design, and more an issue of 
organizational capacity. 9 Elementary schools and smaller schools may provide more 
supportive environments for the implementation of constructivist pedagogy. 

There was less of a “reality effect” for technology skills. The skills that “decreased” from 
Wave 2 to Wave 3 were those that had gained the most initially, and that gained the 
most in the end -- multimedia and Web page skills. Even though technology skills 
improved, there was still substantial demand for further training in these areas, 
especially in secondary schools. 

If we had studied “anticipated use” with students (in Wave 2) and “actual use” (in Wave 
3) we almost certainly would have seen a bigger reality effect concerning technology. 
This is because there is a difference between teachers having technology skills and 
what is required for them to actually use technology with students in the classroom. 
Given the limits of our evaluation data, we have no way of knowing to what extent gains 
in teachers’ technology skills were associated with substantial changes in the amount or 
quality of technology use with students. 10 

The only indication of the quality of computer use with students we have concerns the 
objectives for use that teachers reported. These show an increase in constructivist 
objectives. The fact that these changes were sustained through Wave 3 is perhaps the 
best evidence of a real change in the classroom use of educational technology among 
TWT teachers. 

In conclusion, even though they started out more technologically skilled than the 
average Idaho teacher and more constructivist in philosophy (Appendix A) 
participants in TWT clearly moved even further in these directions. 
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ENDNOTES 



1 For Opportunity 1 results see Opportunity 1 (2002) and Ravitz, Mergendoller and Rush (2002). 

2 For more information about these instruments and the studies where they have been used, see the 
Teaching, Learning & Computing: 1998 project web site -- http://www.crito.uci.edu/TLC 

3 See Appendix B for details about who participated in TWT and who responded to the surveys. 

4 Perhaps these teachers had greater difficulty incorporating technology and saw less need for technology 
to put their constructivist beliefs into practice. 

5 Scores on the constructivist beliefs and technology skills items have been shown to be highly reliable 
and to have predictive power with measures of constructivist teaching practices and professional use of 
computers by teachers (Becker, Ravitz & Wong, 1998; Ravitz, 1999) 

6 If all the teachers from Wave 1 had answered Wave 3 we cannot tell if there would have been more or 
less of a change. It appears Wave 3 teachers held more constructivist objectives initially but still reported 
a number of changes that were noteworthy. 

7 The reader is reminded that conditions for student use may be more favorable in smaller schools and 
lower grades in Idaho, even though teachers in high schools report more skills with technology (Ravitz, 
1999; Opportunity 1 Report; Ravitz, Mergendoller & Rush, 2002). 

8 These findings are consistent with prior research indicating that smaller and younger-grade schools 
provide more supportive environments for teaching with a constructivist-compatible approach (Ravitz, 
Becker & Wong, 1999). 

9 This issue was recently emphasized in A Conversation with Paul Light in the Evaluation Exchange, 
Harvard Family Research Project, 8(2), p. 11. 

10 Prior research shows that technology skills are strongly related to professional use by teachers, but 
less strongly related to use with students in the classroom. Teachers with more constructivist beliefs and 
greater technology skills tend to use computers more frequently with students when other facilitating 
conditions are also present (Ravitz, 1999; Ravitz 2002). 
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Appendix A. Teaching With Technology Survey, Wave #3 

Thank you for filling out this survey in the past. Please fill it out one more time so we can learn 
about any changes that have occurred during the school year. 

1. Name 

First Name? 



Last name: 1 

A. You and Your Job 
A1. 

Name of District: ' 

A2. 

I 

Name of School: 1 

A3. 

Grade(s) you are teaching: | 

A4. 

Subject(s) you teach: ' 

B. Your Teaching Philosophy 

In the past we have asked you to react to statements about contrasting teaching 
philosophies. We are interested in your reactions now that you have had additional 
teaching experiences and time. The following paragraphs describe observations of two 
teachers' classes, Ms. Hill's and Mr. Jones'. Please answer each question below by 
filling in the bubble under the column that best answers that question for you. 

Ms. Hill was leading her class in an animated way, asking questions that the students 
could answer quickly, based on the reading they had done the day before. After this 
review, Ms. Hill taught the class new material, again using simple questions to keep 
students attentive and listening. 

Mr. Jones' class was also having a discussion, but many of the questions came from the 
students themselves. Though Mr. Jones could clarify students' questions and suggest 
where the students could find relevant information, he couldn't really answer most of the 
questions himself. 
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B1 . How do you feel about these two approaches now? 





Definitely i 
Ms. Hill's 


Tend Towards 
Ms. Hill's 


Can't 

Decide 


Tend Toward 
Mr. Jones' 


Definitely 
Mr. Jones' 


a. Which type of class 
discussion are you 
more comfortable 
having? 












| 1 


b 


1 3 


l 4 


j 5 


b. From which type of 
class discussion do 
you think students 
gain more 
knowledge? 












l‘ ! 


1 2 


l 3 


1 4 


b 



B2. Please indicate how much you disagree or agree with each of the following 
statements about teaching and learning, given your recent experiences. 





Strongly 1 
Disagree 


Moderately 

Disagree 


Slightly 

Disagree 


Slightly 

Agree 


Moderately 

Agree 


Strongly 

Agree 


a. Students will take 
more initiative to learn 
when they feel free to 
move around the room 
during class. 














n 


h 


l 3 


1 4 


b 


l 6 


b. A quiet classroom is 
generally needed for 
effective learning. 














| 1 


h 


1 3 


l 4 


| 5 


1 6 


c. It is better when the 
teacher - not the 
students - decides what 
activities are to be 
done. 














i > 


h 


1 3 


1 4 


b 


1 6 


d. Students should help 
establish criteria on 
which their work will be 
assessed. 














h 


| 2 


i 3 


1 4 


b 


I 6 


e. Instruction should be 
built around problems 
with clear, correct 
answers, and around 
ideas that most 
students can grasp 
quickly. 














h '"! 


h 


| 3 


l 4 


b 


1 6 


f. It is very important for 
students to share their 
work outside their 
classroom. 














b 


r 


1 3 


l 4 


1 5 


1 6 
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B3. For the following pair of statements, please indicate how well it matches your 
beliefs. 

"A" I mainly see my role as a facilitator. I try to provide opportunities and resources for 
my students to discover or construct concepts for themselves. 

"B" Students really won't learn the subject unless you go over the material in a 
structured way. It's my job to explain, to show students how to do the work, and to 
assign specific practice. 





Definitely "A" 


Lean toward "A" 


Undecided ] Lean toward "B" 


Definitely "B" 


My beliefs 


1 


2 


3 1 4 


5 



B4. For the following pair of statements, please indicate how well it matches your 
beliefs. 

"C" It is a good idea to have all sorts of activities going on in the classroom. Some 
students might produce a scene from a play the read. Others might create a miniature 
version of the set. It's hard to get the logistics right, but the successes are much more 
important than the failures. 

"D" It is more practical to give the whole class the same assignment, one that has clear 
directions, and one than can be done in short intervals that match students' attention 
spans and the daily class schedule. 





Definitely "C" 


Lean Toward "C" 


Undecided 


Lean Toward "D" 


Definitely "D" 


My Beliefs 


1 


2 


3 


4 


5 



B5. Based on your classroom experiences this year, how useful are each of the 
following kinds of assessments in determining how well students are learning? 





Not 

Useful 


Slightly 

Useful 


Moderately 

Useful 


Very 

Useful 


Essential 


a. Short-answer and multiple-choice 
tests 


b 


b 


b 


1 4 


b 


b. Essay tests 


1 


j 2 


* 


l 4 


b 


c. Open-ended problems 


1 


2 


3 


4 


5 


d. Individual and group projects 


1 


2 


3 


4 


5 


e. Standardized test j 


1 


2 


3 


4 


5 


f. Student ! 

presentations/performances 


1 


2 


3 


4 


b 
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B6. Please mark your TOP 3 objectives for student computer use this year. 



a. Mastering academic skills just taught, or remediating skills 

2 

b. Becoming better writers 

c. Communicating electronically with other people 

4 

d. Finding out about ideas and information 

5 e. Analyzing information 

6 f. Presenting information to an audience 

g. Learning to work collaboratively 

g 

h. Learning word processing skills 

[~9 

I i. Learning computer skills (other than word processing) 

C. Teacher Technology Proficiency 



Cl. Please rate the computer skills you have developed in the following 


areas: 




Don't Know 
How 


Limited: 
Just Learning 


Competent: 
Can Complete 
Satisfactorily 


Expert: 
Can Teach 
Others 


a. Display the 
directory of a disk 


1 


2 


3 


b 


b. Copy files from 
one disk to another 


1 


2 


| 3 


b 


c. Create a new 
database and 
establish fields and 
screen layouts 










| 1 


1 2 


i 3 


1 4 


d. Create a word- 
processed 
document with 
graphics 










‘ 


1 2 


b 


1 4 


e. Create a 
spreadsheet that 
calculates grades 










1 1 


1 2 


1 3 


b 


f. Prepare a slide 
show using 
presentation 
software 










1 1 


1 2 


b 


i 4 


g. Use a World 
Wide Web search 
engine 










| 1 


| 2 


b 


l 4 


h. Create a Web 
page 


1 


2 


3 


4 


i. Troubleshoot 
network problems 


1 


2 


3 


4 


j. Develop a 

multimedia 

presentation 










1 • 


I 2 


| 3 


j 4 


k. Attach files to an i 
email message 


1 


2 


3 


4 



O TWT Evaluation 

ERIC 



18 



21 



C2. How much additional training would you like to receive in the following areas this 
year? 





None 


Start 

From Scratch 


Just 

a Refresher 


Advanced 

Course 


a. Basic computer 
operations (formatting 
disks, finding files, 
etc.) 










| 1 


1 2 


I 3 


1 4 


b. Word processing 


1 


1 2 1 


3 


4 


c. Using spreadsheets 


1 


2 


3 


4 


d. Using presentation 
software (e.g. 
PowerPoint) 










l.‘._ ; 


1 2 


I 3 ; 


1 4 


e. Using databases 


1 


2 


3 


4 


f. Using digital imaging 
(scanners, digital 
cameras, etc.) 










| 1 


1 2 


['"■ 


1 4 


g. Using the World 
Wide Web as an 
instructional resource 










1 1 „ 


1 2 


1 3 


i 4 


h. Integrating 
technology in daily 
teaching and 
assignments 










Ia ! 


1 2 


| 3 


1 4 


i. Creating multimedia 


1 


2 


3 


4 


j. Managing students 
and activities when 
integrating technology 










1 1 


h 


3 


i 4 



D. Comments on the Teaching with Technology Program 

D1. Have the Technology Fellows been helpful to you during the past semester? 

1 Extremely Helpful 

2 

Somewhat Helpful 

H Not Helpful 

D2. Please comment on your above answer. 



A 
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D3. Looking back at your experience, please rate the teaching models discussed during 
the TWT Summer Workshop in terms of how usable you found them to be in your own 
classroom. 





Very 

Usable 


Somewhat 

Usable 


Not 1 

Usable 


Not 

Implemented 


Problem Based 
Learning 


1 


2 


3 j 


1 4 


Learning By Design 


1 


2 


3 | 


4 


Inquiry Based 
Learning 


1 


2 


3 


4 



D4. Please tell us more about your experience with these models. 




J 



D5. Please give a specific example of how you have incorporated one of these teaching 
models into your lesson plans. 



1 



J 

D6. Looking back, has your attitude towards technology changed as a result of the 
Teaching with Technology project? 

Yes 
No 

D7. IF YES, please give a specific example. 



_d 

D8. Email 

Please provide your email address. 




Thanks for taking the time to complete this survey. 



This questionnaire was created using Perseus SurvevSolutions . 
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Appendix B. Who Participated in TWT and the Study? 

It is important to acknowledge that TWT teachers were self-selected into the program. 
This means that they decided on their own to join and that they were therefore “ripe” for 
change in many ways. Teachers who were most resistant to change almost certainly 
did not participate in TWT; it is impossible to know if they would have changed more or 
less than those who did participate. At the start of TWT, those who chose to participate 
were substantially more skilled with technology and they were more constructivist in 
their beliefs than the average Idaho teachers. The following table compares the beliefs 
and practices of Opportunity 1 (Wave 1) teachers who did and did not participate in 
TWT. Data was available for teachers in secondary schools only. 

Comparison of Opportunity 1 Teachers Who Did and Did Not Participate in TWT 



Measures of Beliefs and Practice Z-scores 


No TWT 
(N=355) 


Middle School 

TWT Difference 

(N=60) (ES) 


No TWT 
(N=593) 


High School 
TWT 
(N=63) 


Difference 

(ES) 


Constructivist belief index (higher score = 
more constructivist beliefs) 


-0.08 


0.42 


0.50*** 


-0.01 


0.20 


0.21* 


Constructivist practice index (higher score 
= greater technology skills) 


-0.14 


0.19 


0.32** 


0.06 


0.05 


-0.01 


Mean Importance ascribed to computers 
over 5 years (higher score = more 
importance) 


-0.12 


0.13 


0.25** 


0.03 


0.25 


0.21* 


Technology knowledge and skills index 
(higher score = more skills) 


-0.19 


0.69 


0.89*** 


-0.01 


0.52 


0.53*** 


Teacher professional computer use 
(higher score = greater use) 


-0.15 


0.44 


0.59*** 


0.00 


0.41 


0.41*** 


Teacher use with students [including 
games] (higher score = greater use with 
students) 


-0.16 


0.37 


0.52*** 


0.02 


0.38 


0.36** 



*** p < .001, ** p < .03, * p < .10 

Note. Data is from Wave 1 only and only those who responded to Opportunity 1 surveys (not those who took pre- 
survey in TWT workshops). Mean scores for each grade level were 0.00 with standard deviations of 1 .00. 
Differences for teachers in middle schools were larger than for teachers in high schools. For index scores, see 
Ravitz, Mergendoller, and Rush (2002) and the Final Opportunity 1 Report. 

Throughout the study, we use paired t-tests to control for possible response bias. It is 
plausible that those teachers who responded to Wave 3 were substantially different 
from other teachers who participated in TWT, particularly because Wave 3 was 
administered online. In most cases, we limited our analysis to those for whom we did 
have data in Wave 3. When we compare their initial scores to those who did not 
complete Wave 3, there is a slight difference in technology skills, and no difference in 
their beliefs. Those who completed Wave 3 had significantly more technology skills in 
Wave 1 (ES = .18, p < .10), but did not differ in constructivist beliefs (ES=.03, NS) from 
those who did not complete Wave 3. 

Some teachers may have dropped out of the surveys (not to mention the program) 
because they had less successful experiences. We acknowledge that the precise 
scores that are reported may be somewhat inflated. Instead, we focus more on where 
the greatest changes occurred. 
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Appendix C. Helpfulness of TWT Technology Fellows and Attitude 
Changes, by Wave 1 Pedagogical Beliefs and Technology Skills 



Constructivist Beliefs and Technology Skills, 4 categories 



Grade 



Less skilled with 
computers and 
less constructivist 



More skilled with 
computers and 
less constructivist 
beliefs 



Less skills with 
computers and 
more constructivist 
beliefs 



More skills with 
computers and 
more 

constructivist 
beliefs 



All 

teachers 



All 


Attitude 

changed? 


86% 


55% 


79% 


60% 


70% 


grade 

levels 


Extremely 

helpful? 


82 


65 


58 


76 


71 




N 


28 


20 


24 


37 


115 




Attitude 

changed? 


87 


57 


92 


71 


76 




Extremely 

helpful? 


80 


57 


69 


90 


76 




N 


15 


14 


13 


21 


63 




Attitude 












Middle 


changed? 


78 


60 


57 


75 


69 


School 


Extremely 

helpful? 


89 


100 


29 


78 


79 




N 


9 


5 


7 


8/9 


29 




Attitude 












High 


changed 


100 


0 


75 


25 


56 


School 


Extremely 

helpful 


75 


0 


75 


38 


53 




N 


5/4 


1 


4 


8 


18 



Note. Table shows percents of teachers in each category who indicated “yes” that TWT had an impact on their 
technology beliefs and that TWT Technology Fellows were “extremely helpful.” When there are two Ns the first 
indicates the number of teachers who answered “attitude changed” and the other indicates how many answered 
whether TWT fellows were “extremely helpful.” The numbers of cases are low because many of the categorized 
teachers did not complete Wave 3 surveys. 
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Appendix D. Training Requests by Grade Level Taught, 
Changes from Wave 1 to Wave 3 



Computer 
Training 
Request for... 


Grade 

Level 




Wave 1 
teachers who 
did NOT answer 
Wave 3 
(N=230) 


Wave 1 
teachers who 
DID answer 
Wave 3 
(N=1 10) 


Wave 3 
teachers 
(N=1 10) 


Difference, for 
those who 
answered both 
waves (N=1 10) 


Basic computer 
operations 


Elem 


None 


38% 


27% 


57% 


30% 


Start from scratch 


10 


3 


2 


-2 


Just a refresher 


39 


56 


24 


-31 


Advanced course 


13 


14 


18 


3 


Middle 

School 


None 


37 


50 


67 


17 


Start from scratch 


10 




3 


3 


Just a refresher 


33 


50 


13 


-37 


Advanced course 


20 




17 


17 


High 

School 


None 


37 


50 


56 


6 


Start from scratch 


5 




0 


0 


Just a refresher 


46 


22 


33 


11 


Advanced course 


12 


28 


11 


-17 


Word 

processing 


Elem 


None 


43 


43 


65 


22 


Start from scratch 


1 






0 


Just a refresher 


35 


29 


16 


-13 


Advanced course 


21 


29 


19 


-9 


Middle 

School 


None 


48 


37 


58 


21 


Start from scratch 




3 


3 


0 


Just a refresher 


23 


33 


13 


-20 


Advanced course 


29 


27 


26 


-1 


High 

School 


None 


42 


33 


59 


26 


Start from scratch 


2 






0 


Just a refresher 


22 


28 


12 


-16 


Advanced course 


34 


39 


29 


-10 


Using 

Spreadsheets 


Elem 


None 


14 


18 


31 


13 


Start from scratch 


26 


18 


15 


-3 


Just a refresher 


42 


57 


36 


-20 


Advanced course 


18 


8 


18 


10 


Middle 

School 


None 


10 


7 


13 


6 


Start from scratch 


25 


21 


10 


-11 


Just a refresher 


48 


41 


42 


1 


Advanced course 


17 


31 


36 


5 


High 

School 


None 


7 


17 


18 


1 


Start from scratch 


36 


28 


24 


-4 


Just a refresher 


33 


39 


29 


-10 


Advanced course 


24 


17 


29 


13 


Using 

Presentation 

Software 


Elem 


None 


11 


11 


31 


20 


Start from scratch 


27 


22 




-22 


Just a refresher 


38 


43 


47 


4 


Advanced course 


24 


24 


23 


-1 


Middle 

School 


None 


12 


13 


27 


13 


Start from scratch 


27 


17 




-17 


Just a refresher 


27 


33 


23 


-10 


Advanced course 


35 


37 


50 


13 


High 

School 


None 


17 


17 


28 


11 


Start from scratch 


19 


22 


11 


-11 


Just a refresher 


31 


28 


28 


0 


Advanced course 


33 


33 


33 


0 
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Computer Training 
Request for... 


Grade 

Level 




Wave 1 
teachers who 
did NOT answer 
Wave 3 
(N=230) 


Wave 1 
teachers who 
DID answer 
Wave 3 
(N=1 10) 


Wave 3 
teachers 
(N=1 10) 


Difference, for 
those who 
answered both 
waves (N=110) 


Using Databases 


Elem 


None 


10% 


14% 


17% 


3% 


Start from scratch 


35 


29 


22 


-7 


Just a refresher 


38 


48 


44 


-4 


Advanced course 


18 


10 


17 


7 


Middle 

School 


None 


8 


17 


16 


-1 


Start from scratch 


39 


23 


19 


-4 


Just a refresher 


37 


40 


36 


-5 


Advanced course 


16 


20 


29 


9 


High 

School 


None 


7 


6 


17 


11 


Start from scratch 


43 


28 


28 


0 


Just a refresher 


26 


50 


39 


-11 


Advanced course 


24 


17 


17 


0 


Using Digital 
Imaging 


Elem 


None 


4 


5 


15 


10 


Start from scratch 


34 


32 


12 


-20 


Just a refresher 


36 


33 


46 


13 


Advanced course 


26 


30 


28 


-2 


Middle 

School 


None 


8 


3 


6 


3 


Start from scratch 


51 


50 


6 


-44 


Just a refresher 


20 


23 


41 


17 


Advanced course 


20 


23 


47 


24 


High 

School 


None 


7 


6 


22 


16 


Start from scratch 


33 


41 


17 


-25 


Just a refresher 


24 


35 


17 


-19 


Advanced course 


36 


18 


44 


27 


WWW as 

instructional 

resource 


Elem 


None 


12 


8 


27 


20 


Start from scratch 


16 


13 


11 


-1 


Just a refresher 


41 


37 


37 


1 


Advanced course 


32 


43 


24 


-19 


Middle 

School 


None 


16 


17 


26 


9 


Start from scratch 


12 


10 


3 


-7 


Just a refresher 


43 


37 


23 


-14 


Advanced course 


29 


37 


48 


12 


High 

School 


None 


14 


12 


28 


16 


Start from scratch 


17 


29 


6 


-24 


Just a refresher 


26 


35 


17 


-19 


Advanced course 


43 


24 


50 


27 


Integrating 
technology in daily 
teaching 
assignments 


Elem 


1. None 


7 


2 


13 


12 


Start from scratch 


37 


30 


8 


-22 


Just a refresher 


22 


16 


39 


23 


Advanced course 


35 


52 


39 


-13 


Middle 

School 


None 


6 


10 


10 


0 


Start from scratch 


33 


37 


3 


-34 


Just a refresher 


27 


37 


42 


5 


Advanced course 


35 


17 


45 


29 


High 

School 


None 


12 


6 


11 


5 


Start from scratch 


24 


35 




-35 


Just a refresher 


14 


29 


28 


-2 


Advanced course 


50 


29 


61 


32 
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24 



Computer Training 
Request fo /... 


Grade 

Level 




Wave 1 
teachers who 
did NOT answer 
Wave 3 
(N=230) 


Wave 1 
teachers who 
DID answer 
Wave 3 
(N=1 10) 


Wave 3 
teachers 
(N=1 10) 


Difference, for 
those who 
answered both 
waves (N=1 10) 


Creating 

Multimedia 


Elem 


None 


6% 


5% 


12% 


7% 


Start from scratch 


46 


41 


17 


-25 


Just a refresher 


18 


13 


43 


31 


Advanced course 


30 


41 


28 


-13 


Middle 

School 


None 


6 


3 


10 


6 


Start from scratch 


43 


47 


10 


-37 


Just a refresher 


20 


27 


42 


15 


Advanced course 


31 


23 


39 


15 


High 

School 


None 


0 


0 


17 


17 


Start from scratch 


36 


50 


22 


-28 


Just a refresher 


29 


19 


6 


-13 


Advanced course 


36 


31 


56 


24 


Managing students 
when integrating 
technology 


Elem 


None 


4 


3 


15 


11 


Start from scratch 


45 


36 


18 


-18 


Just a refresher 


21 


21 


39 


18 


Advanced course 


31 


40 


29 


-11 


Middle 

School 


None 


10 


7 


10 


3 


Start from scratch 


37 


47 


7 


-40 


Just a refresher 


20 


20 


48 


28 


Advanced course 


33 


27 


36 


9 


High 

School 


None 


5 




17 


17 


Start from scratch 


42 


35 


11 


-24 


Just a refresher 


15 


24 


28 


4 


Advanced course 


39 


41 


44 


3 




o 

c 



8 
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Appendix E. Overall Changes in Constructivist Beliefs, 
Between Waves, Paired T-test Results 



CHANGES in BELIEFS (Waves 1 -2): After summer workshops (N=196 pairs) 



Effect 

Size 


Direction of 
change 


Type of change 


44 *** 


Constructivist 


Towards Mr. Jones' more constructivist approach, comfort 


.36*** 


Constructivist 


Towards valuing of student initiative in the classroom 


.34*** 


Constructivist 


Students should establish criteria for assessments 


.31*** 


Constructivist 


Towards Mr. Jones' more constructivist approach, knowledge 


-.28*** 


Constructivist 


Away from belief in importance of structured instruction 


-.30*** 


Constructivist 


Away from belief in importance in having a quiet classroom 


-.25*** 


Constructivist 


Away from belief in using simple problems 


-. 20 ** 


Constructivist 


Away from belief in importance of keeping whole class on single task 


-.12 


Constructivist 


Slightly away from belief that teacher should make all the decisions 


.12 


Constructivist 


Slightly toward belief students should share work outside of class 



CHANGES in BELIEFS (Waves 2 - 3): After 10 months of teaching (N=84 pairs) 



Effect 

Size 


Direction of 
change 


Type of change 


0.30** 


Traditional 


Towards belief in importance in having a quiet classroom 


-0.15 


Traditional 


Away from valuing of student initiative in the classroom 


- 0.12 


Traditional 


Away from Mr. Jones' more constructivist approach, knowledge 


0.09 


Traditional 


Away from Mr. Jones' more constructivist approach, comfort 


0.07 


Traditional 


Slightly toward belief in importance of keeping whole class on single task 


0.04 


Traditional 


Slightly towards importance of structured instruction 


0.02 


Traditional 


Slightly towards belief that teacher should make all the decisions 


- 0.10 


Constructivist 


Slightly away from belief in using simple problems 


0.12 


Constructivist 


Slightly towards belief students should establish criteria for assessments 


0.18 


Constructivist 


Slightly towards believing students should share work outside of class 



OVERALL CHANGES in BELIEFS (Waves 1 - 3): 

From before summer workshops to end of year (N=1 14 pairs) 



Effect 

Size 


Direction of 
change 


Type of change 


-0.47*** 


Constructivist 


Away from belief in importance of structured instruction 


0.31** 


Constructivist 


Towards belief students should establish criteria for assessments 


-0.30*** 


Constructivist 


Away from belief in using simple problems 


0.26** 


Constructivist 


Towards Mr. Jones' more constructivist approach, comfort . 


-0.16* 


Constructivist 


Away from belief in importance of keeping whole class on single task 


0.15 


Constructivist 


Slightly towards believing students should share work outside of class 


-0.15 


Constructivist 


Away from belief in importance in having a quiet classroom 


0.12 


Constructivist 


Towards valuing of student initiative in the classroom 


- 0.12 


Constructivist 


Slightly away from belief that teacher should make all the decisions 


08.00 


Constructivist 


Towards Mr. Jones' more constructivist approach, knowledge 



*** p < .001, ** p < .01, * p < .10 

Note. Effect size refers to change in terms of standard deviation. Statistical significance refers to how likely the 
result could have been caused by chance alone. See the survey in Appendix B for complete item wording. 
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Appendix F. Overall Changes in Technology Skills, 
Between Waves, Paired T-test Results 



CHANGES in Skil 


s (Waves 1 - 2): After summer workshops (N=142 pairs) 


ES 




g.j *** 


Develop multimedia presentation 


.78*** 


Create web page 


54*** 


Troubleshoot network problems 


52*** 


Prepare slide show using presentation software 


.34*** 


Display Directory of Disk 


.31*** 


Create wp document with graphics 


.30*** 


Copy Files between Disks 


.28*** 


Attach files to email 


22 *** 


Create Database 


.16** 


Create Spreadsheet to calculate grades 


. 12 * 


Use WWW search engine 


CHANGES in Skills (Waves 2 - 3): After 10 months of teaching (N=57 pairs) 


ES 




.25 


Troubleshoot network problems 


.24** 


Attach files to email 


.18** 


Create wp document with graphics 


.16* 


Copy Files between Disks 


.11 


Use WWW search engine 


.11 


Create Spreadsheet to calculate grades 


.09 


Prepare slide show using presentation software 


.06 


Display Directory of Disk 


-.11 


Create Database 


-.13 


Develop multimedia presentation 


-.31** 


Create web page 


OVERALL CHANGES in Skills (Waves 1 - 3): 

From before workshops to end of vear (N=1 14 pairs) 


ES 




.69*** 


Develop multimedia presentation 


.55*** 


Create web page 


.49*** 


Prepare slide show using presentation software 


47 *** 


Attach files to email 


.46*** 


Troubleshoot network problems 


.37*** 


Display Directory of Disk 


.36*** 


Copy Files between Disks 


.32*** 


Create wp document with graphics 


.28*** 


Create Database 


.12 


Use WWW search engine 


.10 


Create Spreadsheet to calculate grades 


*** p < . 001 . ** p < . 02 , 


*p< .10 



Note. The lowest number of pairs is shown for each set of items. Effect size refers to change in 
terms of standard deviation. See the Appendix B for the actual item wording. 
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